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ABSTRACT 

William Stephenson's Q methodology has been linked to 
the future of communications research by some and condemned as 
treacherous by others. This study reviews the uses of 0 in published 
mass communications research over the past 20 years. The survey found 
3" mass communications studies published in English in the scholarly 
journals related to mass communications from 1953 to 1972. The 
studies used four Q techniques: (1) the Traditional Sort; (2) the 
Modified Free Sort; (3) a "MacLean variation," testing objects rather 
than persons; and (4) Q Factor Analysis. The studies were critiqued 
individually for their performance in five methodological areas; 
basic considerations, item selection, subject selection, distribution 
design, and analytic treatment of data. The results suggest that item 
design and support for the choice of distribution shape are trouble 
spots in the use of Q for communications research to date. 
(Author/TO) 
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ABSTRACT 



Q-METHOD 20 YEARS lATER: ITS USES AND ABUSES 
IN COMMUNICATIONS RESEARCH 
by 

Leonard Freeman, Temple University, Philadelphia, Fa. 

Spring 1974 



William Stephenson's Q methodology has been linked to the 
future of communications research by some, and condemned as 
"treacherous" by others. This study reviews the uses of Q In 
published mass communications research over the past 20 years 
since the appearance of Stephenson's bench-mark text, The Study 
of Behavior; Q Technique and Its Methodology , and provides an 
assessment of Q's strengths and weaknesses for communications 
research on the basis of that review. 

A number of methodological Issues are discussed, with special 
attention being given to the use of parametric statistics, free 
vs. forced sorts, distribution shape, number and makeup of items, 
subject selection, the Importance of building items that "do not 
matter" into sorts, and the relationship between distribution 

The survey found thirty .nass communication studies, published 
in English, in the scholarly journals related to mass communicatlous 
during the years 1953-1972. The studies us(?.d four Q techniques: 
1) the Traditional Sort; 2) the Modified Free Sort, using a question 
nalre format; 3) a "MacLean" variation, testing objects rather than 
persons; and 4) Q Factor Analysis, reflecting Cattell's rather than 
Stephenson's viewpoint on Q. The studies used Q for performing Tive 
basic functions: 1) State of Affairs preferences; 2) Issues prefer- 
ences; 3) Gatekeeping; 4) Congruence; and 5) Prediction. 

The studies were critiqued individually for their performance 
in five methodological areas: basic considerations, item selection, 
subject selection, distribution design, and analytic treatment of 
data. The results suggest that 1) item design and 2) support for 
the choice of distribution shape are trouble spots in the use of 
Q for communications research to date. 



table: of CONTiilNTS 



Introduction 
Method 

Methodologittal Controversy Areas 

Use of Farametrlc Statistics 

Foxced vs. Free 

Distribution Shape 

Item Selection 

Subject Selection 

Criteria for the Assessment of vi-Techni<iue 
Nitn uoflununxuabxua obudieb 

Discussion of Results J Techniques and Functions 

Techniques 

Functions 

Discussion of Results i Methodological Criteria 

Sumiaary 

Footnotes 

Tables 

Appendix At \4. Articles In Commujil-sation Research 

AppendLx Bi Methodologiced Entries on 

Appendix Ci Procedure and Instructions for 
Inter-Coder Reliability 



In 1953 William Stephenson's The Study of Behaviori ^ Tech - 
nique and Its Methodolo,-?:.'/ was -oublished, evoking comments ranging 
I 1 

from "treacherous" to "the most important single contribution of 
the year to assessment," In his book Stephenson outlined the 
practice and rationale for the use of 'xi method in social sciences 
research} and although Q has been the center of controversies, it 
nas Deen largely accepted into the arseha.! of J50cial sciences 
research tools and put to increasing uae. 

In mass communication research, as published in iinglish- 
language journals, Q was slow in being adopted. The first article 
with a clear identification as a ;<i study using Stephenson's approach 
appeared some 10 years later,-' But since that time '^'s use has 
proliferated rapidly. By I965 ^ was being linked to the future 
of communications research} and in 1972 seven articles or researcli 
briefs using Q were published in mass communications journals. 

This study reviews the uses of in published mass 
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communications reseairch over the past 20 years since Stephenson's 
book appeared, and provides an assessment of Q's strengths and 
weaknesses for communications research on the basis of that 
review. Specif Icsdly, the study addresses Itself to the following 
questions I l) What are the Important methodological controversy 
areas with respect to that a communications researcher should 
be aware of in making use of it? 2) What are the primary tech- 
niques and functions used with In mass communications studies? 
3} How have the studies to date performed with respect to the 
aethodologlcal Issues? 4) Are thexeany recurring trouble spots 
where has been consistently misused, or Important matters 
regularly overlooked, in the studies to date? 

Metliod 

The literature was searched extensively for articles and 
research briefs which used (<i- technique , or a clear variation thereof, 
and were published in English-language Journals related to mass 
communications during the years 1953-1972 inclusive. Secondly, the 
methodological literature was searched and the major Issues and con- 
troversies noted and analyzed with respect to communications applica- 
tions* Thirdly, a list of criteria for the assessment of technique 
with communication studies was compiled on the basis of the pre- 
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ceding analysis and a general review of the literature* 
Fourthly, the Q-artlcles were subjected to two types of 
analysis! A) the primary approaches to use of Q with these 
studies were identified and categorized in tenas of 1) tech- 
niques and 11) functions I and B) the criteria for adequate 
use were applied to the articlec^ Individually, and the data 
overall examined to identify trouble spots. 

The reliability of the criteria coding was checked 
through Intercoder agreement. The author's codings were 
used for the analysis in this s^udy. An alternate coder 
was provided with the articles, a set of instructions, (see 
Appendix), and given a brief piractice session using non- 
communications Q studies. The author and the alternate 
coder were in agreement on 79.5^ of the total decision pos- 
sibilities, (208 differences out of 1015 decisions)* 

Literature Search i To find the published reports of 
communications research using Bibliographic Index (Vols. 
4-12) was first searched for the target years, 1953-1972 
Inclusive, under the topics of communication, communication 
research, mass media, and mass media research. Throe basic 
bibliographies "'ere searched i Psycholofcical Abstracts i (VolSc 
.27-46, 1952-1972) A Computerized Bibliography of Mass Com- 
munications Research. 1944-1964 . Danielson and Wilhoit, 1967l 
and Mass Communication i A Research Bibliography . Hansen and 
Parsons, I968. In addition each issue of the following peri- 



id 

ERIC 



odicals was seairched for the entire period i A«V Comroimlcation 
Review (Vols. 1-20, 1953-1972), Gazette (Vols. 1-18, 1955-1972), 
Journal of Broadcasting (Vols, l-17il» 1956-1 97' )» Journal of 
Communication (Vols. 3-22, 1953-1972), Journalism <^uarterl y. 
(Vols. 30-V;, 1953-1972), and Public Opinion quarterly (Vols. 
17-36, 1953-1972). Thirty articles were found. These are identi- 
fied in Appendix A. Twenty-three of the thirty were found in 
one source I Journalism quarterly . 

Toj find the appropriate methodological articles and texts 
Bibliographic Index and the above mentioned three bibliographies 
were again searched. In addition BroHn's "Bibliography on 14 
technique and Its Methodology"^ was searched, along Hith the 
Annual Keview of PsvcholoKy for the entire period under the topic 
"«^", and "Articles on Kass Communication in U. S. and Foreign 
Journals" under the topic of "research methods" in the numbers of 
Journalism i-^uarterly for the period. Other references were taken 
from the articles themselves. The methodol -jical articles and textL 
used in this study are listed in Appendix B. 

Limitations ! Dissertations and other papers not published 
in the Journals noted above were Act included in tne study. 

The critiques were of the research as it was reported in 
the literature . The actual studies may have differed from the manner 
in which they were reported, i.e., data on item design or distribution 
shape choice may be lacking in an article although the author did xn 
fact do good work in these areas. But this could not be assumed to be 
true without support and thus the studies were critiqued as reporrted . 
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Validity ! A rough validation of the Criteria for Assess- 
ment of the Q studies was performed. The author checked his own 
assessment of the relative validity of each criteria item w\th 
those of two communications' professors familiar with <<i method 
and its literature t Drs, iidward Trayes and Thomas Gordon, Temple 
University, Philadelphia, Pennsylvania, A Five point "validity" 
scale was used. 

There was general, positive, agreement on 22 out of the 
25 criteria litems (average score 4.0 or better on the 5 point scale). 
For the three "non-unanimous" items, (items IIil, IIi2, Vi7— see 
pages 23 and 26), the split was 2 to 3 in favor of each Item. 
Item V:7 sconBd the lowest overall average of 3iOt 



Kethodological Controversy Areas 



In the typical ^ study a group of items is collected 
with reference to some particular area of interest. The items may 
be statements, whole or parts of articles, photographs, advertisements, 
headlines or the like— any group of data that caji be combined into 
a. set for sorbin^:; by an 'individual. An individual then sorts the items 
into a forced distribution, usually some variation on the bell-shape. 
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according to a set of instructions such asi "Sort these 
from statements which most describe you to statements 
which least describe you," The resulting Q-array is then 
transformed into a set of standard scores i and correlated 
with sorts by other people who sorted under the same con- 
ditions with the same items i or with another sort by the 
same person on some different criterion. The resulting cor- 
relaticas are then factor analyzed and "typal" Q-arrays 
created for each factor. The statistical treatment may 
varyi but the basic Q-technique is the sort by an individual. 

?he issues of controversy with regard to the use of 
Q-method include i the use of parametric statistics, free 
vs. forced sorts, the shape of the Q distribution, the num- 
ber of items, the criteria for item makeupr and subject 
selection. 

Use of Parametric Statistics 

T?he Q-sort process, as described above, is in essence 
a rank- order procedure. But ranking is normally thought of 
as producing ordinal data for which only nonparametric sta- 
tistical tools are appropriate. How is it then that Q, a 
basically rank-order procedure, uses parametric tools of 
analysji such as Pearson's "r", factor analysis, and "P" 
analysis of variance, when parametric statistical tools are 
usually thought of as requiring interval or ratio data? 

The question has three facets with regard to Qi 
1) the neutral, or distensive, zero basei 2) the equality 
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of intervals I and 3) the adequacy of "approximate" data. 

Neutral zero . Stephenson's rationale and prime claim 
for Q as a powerful methodology capable of using the para- 
metric approaches of factor analysis (and even analysis of 
variance) is rooted in the fact that Q-sorting is basically 
a self-reference operation. "Every measurement involves 
the self expl5citly, as a self concept or the like. " 
•In Q-method . . . all measurements retain self-reference. 

The persons performing Q-sorts usufe-ily sort items 
with reference to themselves, along a continuum from a posi- 
tive (+), through a neutral (0), !;o a negative (-) pole 
(most favorable to least favorable, agree-disagree, etc.). 

9 ^ n . ^ ....... a- J^^^^ J- «4-^ 4-< 0*.^! f^^n^t^ryfyy Y\4* 4 O +1l O 

neutral sector because it is here that one reaches a base for 
transforming the sort from ordinal ranking into interval, 
and even ratio, data reflecting a more solid base in the 
real world and allowing for the use of parametric statistics. 
The neutral point is point zero for transforming the sort 
data into standard scores — "pure numbers whose mean is zero 
and standard deviation io 1.0."^ 

This zero, because it is at a point of not matter- 
ing to the person, is necessarily at the same ab- 
solute level for all persons, for all Q-sorts, for 
all conditions of instruction for a given XYZ situa- 
tion.® 
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Thus, "the scores given to the statements by different Individuals 
are comparable—the zero on all scales is the same absolute value 
for everyone.'" The transfer to pure numbers— standard scores- 
provides the data in intervals, and the basic self reference of 
neutrality for each person provides an absolute zero— the basic 
necessity for ratio data. This establishes a flexible but suffi- 
cient ground for using parametric statistics with ^ data.^^ 

This points up, however, a very important criterion in 
the selection of ^-sort items. There must be neutral items . 
"The ki-saniple is chosen so that many statements do not matter 
(Stephenson's emphasis^ to the subject... "^^ (This Implies in 
turn that one must know something about the persons under study 
and about the items being sorted.) 

If there is not a large number of neutral statements 

in the sort, then in using a "forced" quasi-normal distribution— 

or any forced distribution— one runs the risk of having items 

which do not reflect an absolute zero, for the person sorting, 

12 

being allocated to that position. This then calls into question 
the validity of a study. (For example, if all the items in a 
sort were "live" items for an individual— such as, he got upset at 
any mention of "mother", to which all the items referred— then 
no matter what kind of distribution he or she was forced or 
free to sort the items into, there would be in fact no 
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absolute zero for that person reflected In the study). 

♦++++ 

Equal intervals. Even with. the absolute zero point 
(or ''distensive zero" as Stephenson calls it^-^) there is 
still question as to whether the sort decisions do in 
fact reflect real intervals for the individual sorters i 
and Cronbach and Gleser criticiij s Q for the lack of equal 
intervals. The assumption of equal intervals is a large 
one. There is no question that :he standard scores pro- 
vide neat, clean intervals; the only question is to v/hat 
extent these reflect the real magnitude of the sorter's 
intervals. 

In a Q-study by this author* for example, "^-^ several 
respondents reported that they w«*re much more favorable 
about items they placed at the highest end of their 
forced distribution sort than they were negative about 
items placed at the lowest end of the scale. The impli- 
cation is that a Q-sort may provide a solid reality base 
in the distensive zero of "it do^.sn't matter", but that 
the subsequent intervals on either side of that zero mean 
may not be as precise as one would like. It is thus 
probably appropriate that in the analysis of Q data the ex- 
tremes of item placement receive particular attention, 
while the "in-betweens" are relatively neglected. 
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Approximatio n • The preceding discussion suggests 
that although Q utilizes parametric techniques in treat- 
ment of its data, it is not really producing the kind of 
exact data usually associated with those techniques. Sev- 
eral of the early reviews of Stephenson's Study of Behavio r, 

particularly that by Cronbach and Gleser, attacked it on 

16 

precisely this point. Stephenson and his followers, such 
as S. Brojwn, granted the validity of thoso criticisms if 
one accepted certain mathematical models t but they did 
not accept those models for use with Q. 

Stephenson does not approach the meaning and use of 
the parametric tests commonly used with Q with the same 
expectations of precision as do his critics. In fact one 
might describe his position as the acceptance of approxima - 
tion , rather than absolute precision, as the adequate stan- 
dard for the use of statistics. In Stephenson's view, Q 

is meant more for discovery than for verification ^^— it is 

18 

primarily a bductive rather than deductive. It is a fli> 
id tool--and the statistical techniques used with it are 
intended to be taken and used in a similarly fluid manner. "^^ 

Thus Butler and Fiske recognized that Q-sortings 
Vield no more than a set of ranks with an arbitrary num- 
ber of ties,"^ and that the appropriate statistic for 
correlation would be Kendall's taui but then proceeded to 
move into the realm of parametric tools using the criterion 



I 



Iwf/lO 



of approximation. Product moment correlations "satis- 
factorily approximate** a matrix of tau calculation^ they 
argue I so they can be substituted. The same kind of 
approximation extends one into the use of Thurstone's mul- 
tiple factor analysis, Stephenson readily seems to accept 
this kind of approach to statisticra tools with Q, 

Quite rough and ready procedures are adequate , , • 
since one's real interest is in the psychology i 
and not in anj search for strict parameters or 

the like of sophisticated statisticians,^^ 
It should be clear then that the dpta supplied by Q use of 

Pearson's "r" and factor analysis, etc, does not, and 

should not be expected to, reflect the same precision 

that thoGG parametric tools may elaewliere. 

Even with this proviso it should be further recog- 
nized that some parametric tools may be less appropriate 
than others. The use of analysis of variance with Q has 
long been criticized and/or warned against, Whereas 
the grosser abductive/inductive tool of factor analysis , 
searching for commonalities, can allow for more statistical 
looseness I the more discriminate doductive analysis of 
variance, trying to isolate and identify shades of differ- 
ence, cannot. Despite Stephenson's references to analysis 
of variance, Brown's search of the literature found 

not , , , a single paper by Stephenson in which 

ERIC 
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analysis of variance Is the primary analytic 
method i and only a handful • • • In which he 
mentions (lt)i and then In the form of a war- 
ning that It ought not be given analytic promi- 
23 

nence, 

In view of the previous discussion that warning should be 
heeded. 

If one finds it necessarj; to use analysi- of variance 
with Q, two suggestions by Kerlinger should probably be 
followed, 1) Use a fairly largo number of items in the Q 
sort and 2) raise the requirements for statistical signifi- 
cance—to the ,01 level. In mosc rases of statistical sig- 
nificance with Q that Kerlinger encountered i he found F 
ratios to be so high as to leave little doubt as to statis- 
tical significance. With "borderline" P ratios one would 
probably do well to be cautious in interpreting the atatis- 

2lL 

tlcal significance of Q results. 

Forced vs. Free 

The "forced" nature of Q-data, and in particular 
the fact that they are typically forced into a bell-shaped 
distribution, has been a source of controversy from the 
beginning of Q discussions. Gal to, picking up on Q»s lack 
of statistical independence, reported that "severe defects 
appear present for variance analysis tests of significsnce 
when forced sorting is involved i moderate distortion, v/hen 
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free sort 3s used. And Jon«s. in a direct attack on 
what he saw as the inference implicit in the usual Q-sort 
that "the distribution of the degree of development of the 
universe of traits within individuals is a quasi-normal 
distribution". 26 found in testing three different groups 
that none chose the same free sort and that none chose 
anything close to a bell-shaped curve. Instead they tended 
to roughly approximate different variations on a U-shaped 
curve. It might even be argued that a free sort could 
provide more closely "interval data". (Jones reports 
that his "free" sorters felt confident about their inter- 
vals when queried on the point and thus strengthen Q 

t.hat free 

Btatisxicaiiy . ^^j-a wx v^u.^^ •- - • 

sorts should be substituted for the usual practice of 
using forced sorts with a quasi-normal distribution. How- 
ever the question is not quite that simple. 

The question of free vs. forced procedures in per- 
sonality testing, and particularly the merits and poten- 

tial dangers of forcing. have been discussed at length in 

28 • 

the literature on social research in general. It is a 
two-way street. The primary arguments for forcing are 
that it eliminates "response-sets" which can crop up with 
free procedures, provides data in convenient form for com- 
parison and computation, and encourages the sorter to re- 
veal levels of descrimination that he might not otherwise 
in a "free" setting. On the con side it is asserted that 
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the data thus obtained may be in some sense the "creation" 
of the tool itself I that the "unnaturalness of the forced 
procedure wrecks the spontaneity" of the sorters deoisions* 
chances raising hostility and resistance to the procedure, 
and this increases distortion » and that the loss of ''in- 
dependence" is statistically damaging. 

Even if one decides .0 use a free sort, one must 
decide "how free is free?" Jones "free" sorts, referred 
to above, Actually required the sorters to put at least 
one card into each pile of a nine-point distribution of 
equal -appearing intervals. And I^acLean, in using &. raodi- 
ilcation of free sorts (wherein data secured through a 

■* • ■ 

importance of instructions which "sort of urge them to 
distribute (the items) across (the scale points) and to 
work in a sort of normal distribution." 

What one has to keep in mind with Q is what one is 
trying to do. As discussed above, Q is generally an abduc 
tive tool for discovery rather than verification! thus the 
elimination of response sets and the evoking of more dis- 
criminations are desirable, while the exactness of statis- 
tical procedures is not as essential. Similarly one is 
usually interested in comparability of data, in oommunica- 
tion studies. In a psychoanalytic situation where one 
subject is being tested over time the use of a free sort 



might provide valuable information by the variation in the 

sort shapes, but most communication studies involve the 

study of groups of people and the search for "types" for 

which comparability of data is a primary concern. For this 

the forced sort is generally more convenient. 

On the whole the forced procedure appears reliable. 

Advocates of free sorts have argued that free sorts would 

be more reliable and provide more datai but Block, and 

similarly Prank, found that forced Q-sorts were at least 

as f! table (reliable on re-test) as free sorts, -^^ And in 

addition Block foxmd that the forced sorts produced more 

discriminations and no loss in psychological meaning of 

the tztz. Any "c::tr2 r.ccir.ir.g" thzt z free cert r.ight pre 

duce by analysis of differing sort sliapes, could be found 

in a forced sort by "examination of the Q-item order, i,e,, 

?1 

item content,"-^ 

Whether the free sort mi^ht produce a better inter- 
val is a moot point requiring further verification since 
Livson and Nichols found that the sorters own judgement 
about the "naturalness" of their sorts was highly unreliable 
and basically "irrelevant to reliability of judgement, "^^ 
The ♦bonfidence" that Jones "free" sorters felt in their 
intervals is thus suspect, 'i.he Livson and Nichols study 
further affirmed that "the Qi sort does seem to be able to 
say what the sorter wants to say despite the sorter's 

33 

doubts that his true impressions are •coming through*,"^-' 
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Overall it appears that the use of a forced proce- 
dure with Q is fairly well supported in the literature and 
it is probably more appropriate for the bulk of communica- 
tion studies than a '•free** one. 

Distribution Shape 

The question of the shape also deserves some atten- 
tion. The common practice is to use a normal or quasi- 
normal distribution with little explanation as to why. Yet 
Jones stucly clearly suggested that people would not freely 
choose a bell-shaped distribution.-^ 

A key issue in the question of distribution shape 
is item makeup. The use of a bell-shaped distribution 
iHuao b« aupporueu oy %'nvt xTiem tjexecuion process, ijuiix 
in must be» as Stephenson suggests » many statements that 
do not matter to the subjects and as approximately as 
many for him to agree with as to disagree with.-^^ Given 
this kind of configuration in the item makeup one then 
has a basis for expecting a bell-shaped distribution to 
emerge with many "neutrals" in the middle and roughly 
equal amounts on either end. 

One practice, which appears in the literature occa- 
sionally, is that of basically dichotomizing the items for 
sorting. A strong dichotooiy, especially in a controversial- 
issues area, would naturally suggest a U-shaped distribution, 
rather than a bell-shaped one, on an agree-disagree sort 



Iwf/l6 



for anyone who felt strongly about a particular situation 
or issue. (It would require empirical verification, but 
the question is raised a^ to whether the general "U- shapes 
"freely" chosen by the subjects in Jones' study might not 
have Detjn roughly inherent in the combination of items 
selected for sorting.) The practices of strongly dichoto- 
mizing items for sorting is doubly dangerous because while 
it tends to damage the base of absolute zero of •'not matter- 
ing" (which establishes the comparability of responses) it 
also calls into question the validity of using a bell- 
shaped distribution. Using a forced-normal distribution 
with a strong dichotomy in the items would appear to be 

The building-in of neutral items is thus an important 
part of the support for using a bell-shaped Q distribution. 

It should be noted before we leave this discussion 
of shape that Stephenson rarely uses what is usually thou^,ht 
of as a normal shape — such as (1, 2, 3, ^, 5» ^» 3» 2, 1) 
—but rather suggests, and uses himself, a flattened, platy- 
kurtic shape such as (3, 5» 6, 6, 6, 8, 6, 6, 6, 5, 3). 
The more flattened curve apparently allows for more inter- 
item discrimination. That, according to Livson aiid Nichols, 
increases the test-retest reliability of a sort.-^ Livson 
and Nichols suggestion that as the number of inter-item 
discriminations rises the test-retest reliability will in- 
crease or at least not fall, coupled with Stephenson's sug- 
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gestion that "a miniraura of 10 classes is advisable^ for a 
flattened I platykurtic distributlorvV.i-^^ indicates that a 
bare minimum of piles or classes in the typically odd- 
numbered sort v;ould probably be 9i and 11 a more suitable 
minimum, A flattened (platykurtic) bell-shaped distribu- 
tion should probably be used rather than a strictly "nor- 
mal" one*, In any event the choice of shape should be ex- 
plained relative to a particular study, and supported by 
the structure of the items. 

Item Selection 

The question of how to build r.eutral items into a 
sort raises the question of criteria for item selection, 

Anytning v/nicn neips to expand the variability of 
meaning amongst items, which input j more shades of meaning 
into themi while still keeping them tied to the centi-al 
theoretical/thematic conception of the r.ort, should help 
to build in "neutrals". The obvious procedure for accom- 
plishing this is the balanced-block, factorial design 
which is what Stephenson suggests, "The distensive zero 
is taken care of, usually, v/hun a balanced structured desi^^ 
is employed" in generating sort items, 

In a situation where several variables are being 
built-into a sort, the balancing of these in a conscious 
block design should help to spread out the shades of mean- 
ing of the theoretical variables such that the extreme posi- 
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tions do not dominate and a good many mora nominal posl- 
tions are reflected. Where one does not have the kind of 
theoretical solidity to build a balanced block design one 
is still probably aware of the general components of the 
situation and can include in the items a sufficient num- 
ber of different potential variables related to the cen- 
tral question to mitigate against the U-shaped bias of a 
strong dichotomy, Kerlinger gives the example of a 
study which divided the items up between six different 
"ideal" types of meni theoretical, economic, aesthetic, 
social f political, and religious, -^^ The assumption, or 
hope, i? that for a given personality, one type will 
evoke strong affinity, another strong disassociation and 
the broad remainder reactions somewhere in tiie raiddie. 
One can also help the "normalness" of the shape by pre- 
selecting persons who belong to the types, e,g, , ministers, 
bankers t artists, etc. 

•Phe use of an essentially unstructured sort on the 
other hand may call into question the validity of using a 
bell-shnped distribution and limit a study. If an unstruc- • 
tured sort is the only kind that can be put together, one 
might consider using a "free sort" approach of some type, 
although this will complicate comparisons and computations. 

Another caution in item makeup is to control for 
the social desirability (or health/sickness) of items, 

o 
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According to studies by Edwards and Horst, Edwards, and 
Kogan et al,^^ social desirability of the items definitely 
affects sorting. Edwards found that "one might predict 
fairly successfully, on the average, the Q sorts of sub- 
jects, provided only that we have available the social 
desirability scale values of the items used in obtaining 
the Q sorts. "^^ Kogan et al found that social desirability 
and health/sicknesp were essentially the same variable and 
suggested a way to -partial out" the effects of these 
variables after the sortings. The process is cumbersome 
however and they strongly suggested controlling for the 
factor beforehand in the item selection. 

+++++ 

Two final topics that should be mentioned with re- 
spect to item makeup are reliability and discriminatory 
potential. A good deal lias been said about the stability, 
or sort-resort reliability, of various kinds of Q-tech- 
niques. Quite often it is the variance in the placement 
of items that is tested for reliability j and this can be 
done in a pre-test of the items with a few trial-sorters. 
However, if one goes back to post-test for reliability, 
item reliability may not be as important as overall fac- 
tor reliability. Harris suggests that, clinically, "meaning 
ful relations exist not in overt behavior (the item-content 
of the Q-sorts) but in underlying dynamics (the factors)" •'i 
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and notes that in situations such as psychological self- 
ideal-self sorts over time an individual might "choose 
quite different Q-sort items to express an enduring in- 
congruence between self and ideal-self," Thus in cer- 
tain situations one may want to check for underlying fac - 
tor reliability. 

One place where this may occur is v/ith regard to 
the discriminatory potential of items. Discriminatory 
potential is probably best understood in relation to thcj 
concept 0.^" "consensus," An item is a "consensus item" when 
all the sorters place that item into essentially the 
same position in their Q-arrays, (If, on the other hand, 
an item is placed into a wide range of sorting categories 
by the different subjects, it can be said to have dis- 
criminatory potential, ) When an item is a consensus item 
one possible interpretation is that the sorting group 
shares something in common, and the consensus items can 
be analyzed to identify basic cross-group similarities 
(or factors). 

If the number of consensus items is very high, hov/- 
ever, as it v/as in some of the articles studied, an alter- 
nate interpretation is possible — that the items were not " 
very discriminatory. They v/ere not "sensitive" enough to 
the issue to identify the real differences between sorters. 
The basic validity of a study is in question in such a 
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s ituation I do the results really measure w}v t they say they 
do? Replication with an alternate set of J ap ror under- 
lying factor reliability amongst the sorters would seem ap- 
propriate for settlement of this question in a high con- 
sensus situation* 

How high is "high"? This sur^'ey used a simple 
"guesstimate" guideline that more than 505^ consensus 
might suggest another look at the item makeup, and pos- 
sible replication, depending on the study. A good alterna- 
tive, of course, would be to pre -test the items for thair 

^7 

discriminatory potential. 

Subject Selection 

The choice of subiects can help to assure xnax a 
sort will reflect what it is designed to do. Since the Q 
procedure is usually designed to provide data on personal 
attitude groups, and since it is usually not practicable 
to use Q with populations much over 200 because of the na- 
ture of the techniqu'^, and since it is often useful to 
have some foreknov/ledge of the characteristics of the per- 
sons who will do the sorting, sub j act selection should be 
purposive, rather than random, in most instances. 

In addition the limitations to the generalizability 
of the findings should be noted. V/ith Q one does not 
usually have perfect random samples of some vast universe. 
With subjects, as with statistiosi a rough approximation is 



sufficient for ^'s purposes. What Is essential when one choo&es 
X to be part of the study Is, as Stephenson puts it, "a good 
theory and faith that there are plenty more cases where X came 

On the basis of the preceding discussion, and a review 
of the general literature on '^-method, th =! following "Criteria 
for the Assessment of 14-Technique with Comiaunication Studies" 
have been compiled and applied to the tLi *ty mass communications 
articles using '4 which are reported in Ap-nendix A (pages 5^-57) • 
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Criteria for the Assessment of ^i-Technique with 
Communication Studies 

I« Basic Considerations 

1, The selection of «i for use wjth the study is 
justified. There is some explanation of "why offered 
with respect to the particular study, 

2, Individual persons accompliEb the sorts rather 
than things or groups of people, (I'iacLean to the contraiyj 
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see discussion on pp. 28-30.^ 

3. The similarity of persons is expressed "by a 
correlation based on ipriative scales, i.e., scales on 
which people have distinguiched between items and not 
necessarily scales which distinguish between people on any 
normative basis of individual differences. ^ 

11, Item Selection 

li The items are chosen so that "many statements 
do not matter to the subject, and so that there are as 

50 

many for him to agree with as • • • to disagree v/ith,"-^ 

2. If a forced bell-Rliap^d distribution is used 
the items for sorting are not strongly dichotomous in 

3. A structured design, such as bplanced-block 
factorial design, is used for the selection of sort 
items I and if not balanced-block there is an explanation 
of the item selection process, 

4. If an unstructured sort is used its limitations 
are noted, 

5. The social desirability (sickness/health) vari- 
able is controlled for. 

6. There are a sufficient number of items "for 
stability and statistical reliability^ but not so many as 

to overwhelm the respondents. Probably from 55 to 75 items 
are ideal i"^*'* 6o are usual i^^ and one can probably go aa 



53 

low as ifO If the items are culled from a larger selection,-^-' 
?♦ The items are pre-tested fori 

a) minimum amount of intra-subject variability 
(reliability) 

b) maximum amount of inter-subject variability 
(discriminatory potential), 

III, Subject Selection 

1. The subject selection procedure is described or 
a rationale is provided, 

2, The extent or limitations on the generalis.ability 
of the findings from the subject group is noted, 

IV. Distribution Design 

item design, i,e,, balanced-block design building in 
"neutrals" with bell-shaped j a dichotomous item makeup 
with a U-shapsj a basically unstructured item makeup with 
a free sort, 

2. There is an explanation given of why a particu- 
lar shape was used, 

3. A platykurtic distribuuion is used, rather than 
a strictly "normal" one, to enhance interitem discrimina- 
tions. 

There are a bare minimum of 9 sorting piles, or 
categories (but hopefully 11 or more). 
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V, Analytic Treatment of Data 

1. The statistical limitations of Q-data per se, 
are recognized, (i.e., that Q data is not strictly para- 
metric i or that the parametric correlation coefficients 
and factor analysis and/or analysis of variance are pro- 
viding only approximate data when used with Qi or that 
•♦independence" is lacking in the decision-making, etc.) 

2. The statistical tools for analysis are clearly 
identifiedi, (Pearson's "r" and factor analysis are usually 
used and Stephenson recommends hand-rotation of factors. 

If some other tools or techniques are used their f<rp^opri- 
ateness for use with the study should be discussed. ) 

3. Analvsis of Variance is not nr>rm«ny uRf»rJ be- 
cause of lack of independence in the data and difficulties 
in establishing the equality of intervals and homoscedes- 
ticity. 

4. IF ANALYSIS OP VARIANCE IS USEDi 

a) the level of significance is ,01 or higher 

b) a larger number of items is included in the 
sort I (cf. Item Selection II t6 above— probably 75 or so is 
sufficient, ) 

5. The factors are analyzed in terms of the postu- 
lated "theory" built into the set of items. 

6. If reliability was not established in pre-testing, 
there is aomo post-test measure of it. 
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7. Depending on > study, more than 50^ "consensus" 
items suggests a relook -he item makeup and possible re- 
plication with alternate ,ems for verification through 
"factor reliability.". 

Discussion of Result i Techniques and Functions 
Techniques 

Thirty articles using Q or a clear variation thereof 
were published in English in the mass communications re- 
lated journals during 1953-1972. The majority of articles 
followed the basic Q-sort pattern with persons sorting items 
into some kind of distribution on some given criterion. 
(These are marked with an "S" in Table 1). However at 
leant three significant variations on the basic process 
occurred. 



Table 1 about here 

First, six articles (McGuire 19, Atwood and MacLean 
22, KacLean et al 25 1 MacLean and Hazard 30 Harless 6 & 7) 
used a Modified-Free-Sort (MF3). In this approach persons 
do not sort items into a forced distribution on a compara- 
tive basis, but rather respond to a questionnaire (usually 
by mail) rating questionnaire items on some numerical scale 
(0 to 9 for example). The questionnaire data are later trans 
lated into a Q matrix/array and factor analyzed by the re- 
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searchers, MacLean noted that in some cases this method 
yielded highly similar results to the typologies gathered 
using the traditional Q-sorts.^^ He stresses that when 
using this approach it is important to provide instruc- 
tions which encourage comparative responses to the iteiiis 
and which start the subject "thinking in a normal dis- 
tribution pattern", such as i 

You'll probably find a few of these which you 
will want to agree with very strongly and a few 
which you will disagree with very strongly. 
Probably most v/ill be intermediate for you. 
Read through all the items before you begin to 

^6 

lation to one another.-^ 
This approach has basic roots in Q. However, there are 
potential differences, The MPS uses a "free sort" pat- 
tern and relies on getting a "normal" distribution by 
virtue of the instructions and by chance. Also the item 
responses are probably not as directly comparable to 
each other as are "sort" responses. Hov/ever the approach 
has the basic advantagrjs of being generally faster and 
easier to administer i the study data can be secured by 
mail, and there can be a larger number of respondents. 

When there are f.x large number of respondents, how- 
ever, the basic factor analysis in an MF3 study is often 
perfoi-irted on only a portion of them. In the MFS studies 

ERIC 



lwf/28 

by MacLftarii for example, the total respondents ranged 

from 30^ to 637 {.^22, 25, 30). However the number actually 

analyzed to identify factors, was similar to that used in 

a typical Q-sox-t operation 1 100 or so. The additional 

57 

respondents were later correlated with the factors.^' 

A second variation suggested by MacLean for work- 
ing with Q in communication studies seems inappropriate. 
This is his suggestion that '*in place of persons, we might 
use newspapers, communities, counties, nations— in fact, 
any units for which we can obtain systematic, codable des- 
criptive data."^^ This sounds ent. '.cing— an application 
of Q to such things as content analysis for example. The 

« Trnr^z-io nVi won ry^rtXrat^ »m V\ir Too— Wr»y> T.ais f 0 ^ ■Pr\Y* Q r>Ar^ + Av^ + 

analysis of "Editorial Support and Campaig-n News." His 
study seems to demonstrate the fallacy of this kind of ap- 
proach to Q. The basic problem is that when data from 
some source other than a person 5s used, the absolute 
zero value, that Stephenson insists makes the scoring 
of different individuals comparable, is lost. V^ith people 
one can assume that "the zero on all scales is the same 
absolute value for everyone"^^ — the point of not mattering. 
But how can one assert that about the content of a news- 
paper for example? What gets into newsprint is by defini- 
tion something that matters. And the Lee study in fact 
attempted to "reduce the neutral category by making the 
assumption that exposure is a pcaitive value. "^^ When 
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such data is translated into a Q-array (a set of scores), 
the supposition of a neutral pile is hard to maintain. 

In Lee's article there is no guarantee, and good 
reason to doubt, that the intervals between the piles in 
the Q array would be equal. With no zero point for com- 
parison and the lack of an equal interval the data are 
clearly reduced to the level of ordinal ranks. 

The basic fallacy is to assume that "newspapers are 
complex individuals with their own particular personalities" 
as Lee does, or to assume the same thing about communi- 
ties or counties or the like. It is handy to anthropo- 
morphize non-human items and aggregates of people for ease 

of onmmunios tl on hut Orxnct fssll in+.rt +r»»n-h^r»nr +hnr>o;c! AV 

groups as individual people we have fallen for our o\m 
rhetoric — and taken analogies for truths. 

V/ith a sort by people the items are extraneous to 
the Gorters--at least in the immediate test situation— 
and they can respond through a!!l the degrees of attitude 
from pro through neutral to con. However, data that com- 
prise both the items list and th^. "sort scores", have no 
neutrality. All of the items do matter in terms of the 
something, or else they would not be part of it. And the 
simple absence of an item from a source cannot be taken 
as meaning that it doesn't matter. In the example of 
the newspapers, the omission of some item from a given 
paper might reflect a negative bias such that the paper 
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intentionally omits it. The fact that the question of 
which omitted items "count" and which are of ♦'no matter" 
can be seriously raised, indicates that a solid, com- 
parable zero point among the "sorters" has been lost. 
It should be clear that there are serious deficiencies 
with this approach to Q. 

Sfdies by Donahew and Singh (1^), and Grunig (15) 
represent a third variation on Q which appeared in the 
literature. These studies differed significantly from 
the usual Q process. Rather than comparative ratings of 
items in 0 sort, or on a questionnaire form, these two 
studies ubilized a series of test administered to the 
subjects. In Grunig «s for example subjects were asked 
why they did or did not perform a particular activity and 
then their responses were coded b^r the researcher into one 
of four "decision categories". An average score for each 
category vas determined, and the scores for the total 
group werp factor analyzed. A considerable amount of 
normative data (expenses, taxes, etc.) was also collected 
and utilized. Donahew and Singh's study was similaro 

While the subjects were analyzed into "types", and 
the data reflected some attitudinal preferences on the part 
of the subjects, those appear to be the only concrete con- 
nections to Stephenson's Q-method in these studies. The 
subjects were essentially rated for comparison. They did 
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not rate other things in a comparative manner. No dis- 
tribution, forced or free, of comparative discriminations 
was involved » and the comparability of the subjects* data 
was certainly not based on any distensive zero point of 
neutrality, or '•it doesn't matter," 

For these reasons these studies probably do not be- 
long in this survey, Donahew specifically disassociated 
his study from Stephenson's use of Q to align with the 
"Q" espoysed by Cattell,"^^ However it was felt that all 
studies which identified themselves as "Q" in the litera- 
ture should be included for comprehensiveness and compara- 
bility and to avoid confusion. For purposes of identifi- 
cation these two studies were labeled with Catell's termi 
Q Factor Analysis (QFA), 

With respect to the analytic treatment of data, the 
primary method was some form of factor analysis, Stephen- 
son recommends hand-rot? tion of factorsa to allow for a 
more subjective, abductive approach to the data. But moirc 
of the studies herein used some other approach. Two stuaies 
(12, 19) used linkage analysis. According to KcQuitty, 
linkage analysis "sometimes yields results very similar to, 
or identical with, those of rotated factor-analytic solutions 
Its primary assets are speed and objectivity. In addition a 
large number of studies made use of a computer program, 
QUANAL, or some sub-part thereof, developed by Norman Van 
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con mmy^ 

Tubergen of the University of Kentucky with Vincent Parace 
and Malcolm MacLean. * 

Functions 

Q method was used in the mass communications litera- 
ture to perform essentially five basic functions reflect- 
ing five basic questions. The functions i questions i and 
articles are identified below. Some articles used Q in 
more than one way, 

1) State-of-Affairs -Who thinks what about XYZ 
(a state of affairs i practice, or thing) and why?" 

1. 3i ^9 5i 9, 19, 22, 25, 30. 

2) Issues "Who has what preferences or attitudes 

of XYZ and why?" 

6./7, 8, 9, 12, 20. 

3) Gatekcopinfi "On what basis do they decide to 
use XY2 or not?" 

10, 13, 16, 17, 21, 29. 
^) Congruence "Does XYZ really reflect what ABC 
believes or prefers?" "How does ABC compare with himself 
on another sort?" 

2, 11, 23, 24, 26, 27, 28. 

5) Prediction "Who is best at predicting XYZ's 
preferences?" 

10, 18, 21, 27. 
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State-of-Affairs and Issues studies were very much alike 
and together reflect half of the studies (9+6 =15). These 
all involved establishing "typologies" of persons, Harless 
(6/7) used this approach to determine groups for experimen- 
tal study. 

The GatekeepimT studies typically involved a magazine 

or newspaper and how the editors, or hypothetical editors, 

would determine which articles, pictures, etc, to include, 

Clyde and Buckalew quote MacLean to the effect thati 

The sorting procedure closely resembles the 

gatekeeper decision process. This is because 

the editor compares all the items in a given 
'^..t •.1....... - «». • » 

Editorio.l decisions were not the only use for this format 
however I Ellingsworth (29) tested the usefulness of various- 
sections of a newspaper for teachers' classroom use. 

The Con.Kruence studies wore i?ome of the most inter- 
esting surveyed. Two of the experimental studies (11 & 26) 
which used Q as the primary technique and analysis of vari- 
ance involved this approach. This function was also used 
in conjunction with content analysis (2, 23, 23, 28) j d 
duo which provides solid data for beefing up the "so what" 
which can greet so many content analysis efforts. Ferrule's 
use of this approach (26) was closest to the classic psycholo 
ical self-ideal-self use of Q, 



The Prediotion studies are fairly self-explanatory. 
The basic thread in most was whether the people who are 
supposed to be predicting other people's preferences 
(i.e., editors) are any good at it. MacLean (2?) portrays 
a novel approach for improving an editor's prediction quo- 
tient. 

DDScussion of Results i Methodological Criteria 
Almost all of the Q studies surveyed satisfied the 
basic ccnsiderations. However, three studies, all of 
which used variations from the basic sort procedure, viola 
ted one or more of the basic considerations. Lee's {?) 
adoption of WacLean's suggestion, using things rather than 
retil jjex-Bonu Cor accompiiohing sorxs, nas already Deen 
criticized, (cf. pp. 28 & 30 above). Donahew and Singh 
{ll^) and Grunig (15) were also previously discussed (p. 
25 above) and their differences from basic Q noted. Both 
used normative as well as ipsative data. They inherently 
did not build neutral elements into the data. 



Table 2' about here 

Use of a balanced-block design of some type was 
accepted as working to build in "many statements that do 
not matter" etc. on the basis of Stephenson's affirmation 
that such a design would usually take care of establishing 
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the dls tensive zero of the sort* Basically unstructured sorts 
were taken as not working In this manner. It should be noted 
that none of the authors studied explicitly stated that they were 
In fact trying to build such a pattern into the ixems* 

On the whole a sufficient number of items were used in most 
studies to raise no questions about the instrument's ability to 
evoke a sufficient number of discriminations— similarly with the 
number of piles used. Those studies which were weak in these 
areas can be noted In Tables 1 and 3> 



Table 3 about here 



sufficient detail (21 out of 30 articles) but there were very few 
comments about the limitations on generalizabllity, In fact about 
half of the studies, l6, rated only minimally 0^ ; and this only 
because they id not extend their findings beyond the group studied 
or used tentative language such as "the data suggest that some 
XYZ's feel thus and so," These were accepted as implying limita- 
tions, A few however were quite specific, notably MacLean ffJO 
and f/25, Rucker r'/20, and Cabhcart //12, Those who were rated neg- 
atively (x) on this criterion appeared to generalize their findings 
without noting any limitations. 



Table 4 about here 



The statistical limitations of Q data were essen- 
tially ignored or unrecognized by the bulk of the studios 
(24 out of 30). This is unfortunate since the studies ap- 
pear in scholarly Journals where other studies use the 
same statistical tools with assumr. iions of parametric 
precision tnatar^not warranted in the case of Q. 

Static Ijlcal tools were gene.'^ally identified, al- 
though in a few cases little was G.r,id except tliat inter- 
correlations and factor analysis wore performed. This was 
not considered sufficient since it is theoretically pos- 
sible "oo use non-parametric factor analysis v/ith Q^^ and 
there are various rotational procedures t^c, which might 
evoke differrjuL kinds of factors, In any event replica- 
tions and data verification checks would not be possible 
with such skimpy information. 



Table 5 about here 

On the whole it appears that, in communications 
studies at least, the warnings to stay away from analysis 
of variance with Q are generally being heeded, Orily two 
studies used analysis of variance and both on the whole were 
quite well done (although Brown //II had a relatively small 



.lwf/37 



item saraple). They noted statistical limitations, par- 
ticulfirly rei independence of sorts, and both used the 
higher ,01 significance level. Both were experimental de- 
signs, Perrullo's (#26) was in fact one of the best de- 
signed and articulated studies surveyed with a very good 
introductory section on Q, methods 

The major, consistent, trouble spots have to do 
with item and shape design and the question of validity, 

Tv/o-thirds of the studies surveyed clearly built in 
some sort of structure to their xtera design (cf. Table' 11), 
Thus, in communication studies At least, items are not 
gen'jrally just "slapped together as Milholland criticized,^'' 
However when an unstructured desig-n was used there v/as no 
discussion of limitations which this might potentially pre- 
sent for a study* This absence of comment we would suggest 
as being related to a major problem--that a relation be- 
tween distribution s}iape and itei; design is not generally 
recognized— or at least not verbalized. None of the studies 
surveyed contained any refeiftnco to such a relationship 
and in ^ir out of 15 of the studies for which a clear yes/no 
answer was discernible, there appeared to be a conflict be- 
tween item design and the shape chosen for the distribution 
(cf. Table 6 ), In judging the articles t^e use of a non- 
dichotoinous structured sort was considered as providing 
"item-design support" for a foii'ced bell-shaped di^^tribution, 
Rucker's study (//20) was rated marginally acceptable in 
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this area. Although he used a basic dichotomy in the item 
design with an issue which was strongly dichotomous, he 
also balanced in other themes of relevance to the topic 
which probably served to mitigate against the sharpness 
of the dichotomy, A glance at Tables 1-6 indicates 
that, on the whole, the design of his study was quite good, 

Tata.e 6 about here 

It should be noted that none of the studies reported any 
conscious attempt to build "neutrals" into the sort nor 
any recognition of a relation between the structure of the 
item design and the shape of the distribution utilized. 

Only one study, (Ferullo ZC), offered any explana- 
tion of why a particular distribution v/as adopted. Of 
those using some variation of a bell-shaped ^urve, only 
two researchers came close to using a platykurtic dis- 
tribution one of whom was Stephenson himself. It appear;^ 
that on the whole V/ittenborn»s criticism that the quasi- 
normal shape is used generally, and without much thought 
as to why, seems warranted. As was noted earlier, dis- 
regard for the link between shape and item design runs 
the risk of damaging the validity of a study. 

The social doairability of the items for sorting 
(Table 3 ) was not "lentioned in any of the studies and it 
can only be assumed that on the whole it was probo.bly 
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never adjusted for, either before or after the sortings. 
This, if Edwards et al» are correct— and this survey found 
nothing in the literature to suggest that they aren't-- 
is a serious omission and potentially calls into the .ques- 
tion the validity of the studies. 

Reliability is apparently all too often ignored ii» 
Q studies (Tables 3 & 5), Only one of those studies 
(Flynn //8) did an item reliability pre-test and none re- 
ported any post-test reliability check. Similarly the 
question of the discriminatory potenbial of the items was 
generally left untested; and in some cases it was an imp(tr- 
tant question. 

With respect to the level of consensus (all sorters 
sorting an item into essentially the same position) the 
studies ranged from a low of zei^o consensus (Ellingsworth 
//29) to a high of 76?S consensus in the studies by Clyde and 
Buckalew (#16) and Buckalew (//I?) (Table 5). Both of these 
appear to have used the same set of items, and 49 out cl 
the 64 items were found to be consensus items in each c£/,se. 
As noted earlier, one has to ask whether such a result 
proves that the subjects were basically alike--or simply 
that the items were not very discriminatory, (It is to 
the authors' credit that they called attention to this ques- 
tion themselves, )^^ Replications of the aforementioned 
two studies should probably be performed before their re^ 
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suits arc relied upon. 

+++++ 

It appears on the basis of the precedlno discussion, that 
the issuo of what would be generally termed validity is a major 
trouble spot vdth 4 studies in communications, Most of the serious 
problem areas have to do with item construction and shape of the 
distribution used, Control for the social desirability (sickness/ 
health) of items seems to be the most regularly overlooked factor 

in item desisn, and a serious and potentially damaginf; omission in the 

I 

communicatipn studies using 4 published thus far, Secondly the 
apparent lack of awareness of the relation between item design and 
distribution shape, which su^^ests a lack of awareness of the 
importance of "neutrals" in .i-corts, ic the major problem in the 
tcchiiica,l use of "Lliu mwthod. 

Apparently the majority of comiuunication studies usin^ 
methodoloj^y to date have relied lar^;;ely upon a kind of "face" 
validity— basically assuming that "the thiHi^- just nejms what it says 
it means" and that people vfould just "naturally" sort thino;s into 
a "noriual" distribution, A rc'iew of the literature however, su^oscts 
that such assumptions about the workings of .-^-method really cajinot, 
and should not, bo made witliout support. It is time that ^ studies 
in mass cor.uiunications research started to consciously build some 
validity support into their designs and analysis. 
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Siunmary 



This study set out to review the uses and abuses of 
Q method in published communications research since the ap- 
pearance of Stephenson's bench-mark texti The Study of 
Bohavior . twenty years ago, 

A number of methodological issues were discussed. 
The use of parametric statistics, i.'ree vs. forced sort, dis- 
tribution shape, number and makoup of items, and subject 
selection, the importance of building items that "do not 
matter" into sorts, and the relation between distribution 
shape and item design were given particular attention. 

The survey found thirty ml as communications studies, 
published in English, in the scholarly journals related to 
mass communications. They usftd four Q techniques i 1) the 
T.raditional Sort, 2) the Modified Free Sort using a ques- 
tionnaire format, 3) a 'WacLean" variation testing objects 
rather than persons, and ^) Q Factor Analysis, reflecting 
Cattell's rather than Stephenson's viewpoint on Q, The 
studies used Q for performing Hive basic functions i 1) State 
of Affairs preferences, 2) Issues preferences, 3) Gatekeep- 
ing, ^) Congruence, and 5) Prediction. 

The studies were critiqued individually for their 
performance in five methodolosic??l areas i basic considera- 
tions, item selection, subject selection, distribution de- 
sign, and analytic treatment of data, Methodologically 
the prime trouble spots were in 1) item design and 2) sup- 
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port for the choice of distribution shape. Validity should 
be a watch~v/ord for future users of Q« 

Overall Q-method appears to be a tool v/ith a variety 
of uses for social rcriearch in mass communications; but 
this study suggests that its operative design must be proper- 
ly constructed and the. limitations of its statistical pro- 
cedures clearly noted» if it is to be used most fruitfully. 
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Techniques and Supplementary Data on 
Communication Azrbicles Using Q 
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Table 2 

Criteria Analysis-Basic Considerations 



1 2 3 

Justification Individual's Correlation 
of Q use sort based on 

ipsatlve scales 



1- Ruffner * « # 

2- Lee * X X 

3- Van Tubergen et al * ♦ * 

4- Sanders * ♦ * 

5- Meyer r x ♦ * 

6/7-Harless * * * 

8- Flynn * ♦ * 

9- Larkin x * ♦ 

10- Atwood * * * 

11- Brovm * « * 

12- Gathcart * * * 

13- Patterson et al x ♦ ♦ 

14- Donahew et al * * x 

15- Grunifi * « X 

16- Clyde et al ♦ * ♦ * 

17- Buckalew * * * 

18- Atwood ....a..., * * * 

19- McGuire x * ♦ 

20- Rucker * * * 

21- Lassahn .«.« x * * 

22- Atwood et al x * * 

23- Clarke et al x * ^ 

2^-Bomholdt * * *• 

25- i'iacLean et al * * * 

26- Ferullo * * * 

27- MacLean et al * * * 

28- Stephenson * * * 

29- Sllingsworth x * * 

30- MacLean et al ».. - * * 



Key y 

♦ ■ Yes or present 
** ^ Conditionally acceptable 
X ■ No or not present 
- ** Not applicable 
? ■ Data unclear or unspecified 
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Table 4 

Criteria Analysis-Subject Selection 



1 2 
Procedure Limitations 
described noted 



1- Ruffner ( 

2- Lee ( 

3- Van Tubergen et al 

4- Sanders , 

5- Meyer , . . , 
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8- Flynn , 

9- Larkin , . . < 

1 0- Atwood , 

11- Brovm , 

12- Cathcart < 

13- Patterson et al 

14- DonaheK et al 

15- C'runlg ..( 

16- Clyde et al 
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APPEI^DIX A 

Q ARTICLES IN COrOTNICATION RESEARCH 

Articles are listed chronologically, 
* « Research brief, 

1972 1-Ruffner, Marguerite Anne, "Women's Attitudes Toward Pro- 
gressive Rock Radio", Journal of Broadcasting 17ili85- 
9^, (State of Affairs*) 

2- Lee, Jae-won, "Editorial Support and Campaign Newsi Con- 

tent Analysis by Q-Method", Journalism Quarterly ^9i z+i 
7IO-7I6, Followed Maclean's (#26) suggestion and used 
newspapers as "persons" and their performance scores in 
content categories as "test items", (Congruence) 

3- Van Tubex^gen, G, Norman and Karen Priedland, "Preference 

for Comic Strips Among Teenagers", Journalism Quarterly 
491^1 7^5-750, (State of Affairs), ^Unstructured; 

^-Sanders, Keith P. "Q Study of Editors* Attitudes Toward 
JoumalisTi Research", Journalism Quar terly 49 1 3i 519- 
530, (SUte of AffairiX 

5-Meyer, William G, "Q-Study of Attitudes Toward Rock Fes- 
tival in Iowa Tov/n", Journalism Quarterl y 49 1 2i 351- 



6- Harless, James D, "The Impact of Adventure Fiction on 

Readers I The Nice-Guy Type", Journalism Quarterly 
2t 306-315, See number 7, 

7- , "The Impact of Adventure Fiction on 

Readers I The Tough-Guy Type", J ourrmlism Quarterly 
491 li 65-73. Q v/as used for assignment of subjects 
to exper/.nental groups. (Issues), 

1971 8-Flynn, Jaires H, III, "The Ideal Television Station 1 A 
•Q* Study", Journal of Broadcasting 16 1 li 65-77, 
(Issues )• 

9-Larkin, Ernest F, "A Q-Analysis of Values and Attitudes 
Toward ASvertising", Journalism Quarterly l^Bt It 68-72. 
(State or Affairs I Issues)";; 

1970 10-Atwood, Srwin L, "How Newsmen and Readers Perceive Each 
Others* Story Preferences", JouiTial i^'m Quarterly ^7i 
2 1 296-302, Editor were poorer pred:Lctors than were 
the other staffers of audience preferences, (Prediction! 
Gatekeeping), 



ERIC 



ll-Browrii Steven R, "Consistency and the Persistence of 

Ideology I Some Experimental Results", Public Opinion 
Quarterly 3'+! li 60-63, Comparison of Q sort re- 
plications over time. (Congruence) 

1969 12-Cathcart, V/illiara L, "Viewer Needs and Desires in 

Television Newscasters", Journal of Broadcasting: 
l4i It 55-62, Used linktige analysis and unstruc- 
tured sort 4 ( Issues }• 

13-Patterson, Joye, Laurel Booth and Russell Smith, "Who 
Reads about Scienou?" Journali sm Quarterly ^61 31 
599-602, (Gateiteeping),* 

iJ^-Donahew, Lewis, and B, Krishna Singh, "Communication 

and Life Styles In Appalachia"j Journal of Oomniuni ca - 
tion 191 3 1 202-216, Cattell»s Q not Stephenson's, 
Identified typos of individuals, 

15- Grunis, James E, "Information and Decision Making in 

Economic Development 1 Journalism Quarterly ^oi 3i 
5^5''575» Design similar to Donihew's, ./l^^ 

16- Clyde, Robert W, and James K, Buclcalew, "Inter-Kedia 

• Standardization! A Q-Analysis of News Editors", 
Joumaliarn Quarterly ^16 1 Z\ 3^9-351. ^9 out of 

disuriminaLlng itema? (Gat«ktsepin^),* 

17- Buckalow, James K, "A Q-Analysis of Television Edi- 

tors' Decisions", Journal ism Quarterly ^61 li 
135-137. Probably used samo itoms as'?l6, ^^9 
consensus items out of 6^ raises same question, 
(Gatekeeping),* 

1968 18-Atwood, L, Erwin, "Perception of Television Program 

Preferences among Teenager!^ and Their Parents", 
Journal of Broadcast inj^ 1Z\ h*. 377-388, (Predic- 
tion } , 

19-McGuire, Delbert, "Democracy's Confrontation 1 The 
Presidential Press Conference, TI", Journalism 
Quarterly ^^5» It 31-5^1-. Attitudes of v/hite House 
correspondents toward tJie practices for presidential 
press conferences, MPS Linkage analysis, (State 
of Affairs), 

1967 20-Rucker, Br^ --^ W, "V/hat Solutions Do People Endorse in 

Free Pre ..-Fair Trial Dilemma?" Journalism Quarterly 
kki 2 1 2^0-2^^, (Issues), 

21-Lassahn, Pamela Henry, "Comparison of Judgements, about 
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1966 

1965 
1963 



1953 



Agricultural Science News", Journalism Quarterly 
kUt 2 1 702-707t (Prediction I Gatekeeping). 

22- Atwood, L. Erwin and Malcolm S. MacLean Jr. "How 

Principals, Advisers, Parents and Pupils View 
Journalism", Journal if] m Quarterly ^^l-i It 71-78 • 
Used mail questionnaire. (State of Affairs). 

23- Clarke, Peter and Virginia Esposlto. "A Study of 

Occupational Advice for Women in Magazines" Jour - 
nalism Quarterly ^3» 3» ^77-^''Q5t Used Q sorts of 
readers » preferences as comparison against the find- 
ings of a content analysis. (Congruence). 

24- Bomholdt, John N. Jr. "Should the Student Press Be 

More Serious?" Jounialism Quarterly 43 1 31 560- 
562. Preference types compared vath a content 
analysis. (Congruence).* 

25- MacLean, Malcolm S. Jr., Thomas Danbury and John T. 

MoN'elly. "AEJ Members and Their Attitudes on 
Journalism Education Issues". Journalism Quarterly 
k2t It 98-107. (State of AffairsTT 

26- Ferullo, Robert J. "The Self-Concept in Communication", 

Journal of Communication 13 1 2i 77-86. Q-sorts 

and personality traits of 'better and poorer speakers. 
One of the better designed and describecj studies. 
(Congruence). 

27- MacLean, Malcolm S. Tr. and Anne Li -an Kao. "Picture 

Selection I An Editorial Game", Journalism Quarterly 
J^Oi 2t 230-233. The editors* ability to "predict^ 
the preference of the "average reader"— before and 
after exposure to the readers sorts, ( Congruence | 
Prediction).* 

28- Stephenson, William. "The 'Infantile* vs. the •Sublime* 

in Advertisements", JournaliGm Quarterly 40i 2i 
181-186. Comparison of reader preferences to a con- 
tent analysis. (Congruence). 

29- Ellings\vorth, Huber W, "Teacher Preference for News 

Items Used as Class Materials", Journalism Quarter - 
lil kot it 87-93. (Gatekeeping). 

30- MacLean, Malcolm S. Jr. and William Hazard. "Women's 

Interest in Pictures 1 The badger Village Study", 
Journalism Quartorly 30 » 2i 139-162. Not strictly 
Q but an early form in which the rudiments can be 
seen for testing pictorial data fcr interest and 
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factoring for types. Good explanation of procedure. 
Forerunner of the "rated data later turned into a 
Q-array« (MPS) type of study. (State of Affairs), 



APPENDIX B 
I^THODOLOGICAL ENTRIES ON Q ^ 

1- AnaGtasi, Anne, "Individual Differences", Annual Rftvlevv 

of Ps.vcholo;cv . 137-156, (esp, pp. l'l^-1^5)i 1953^ 

2- Block, Jaclti ♦'A Comparison of the Forced and Unforced 

Q-Sorting Procedurea", Educational and P3.ycholo/>.ical 
Measurement loi ^8l-.'+93, 1956, 

3- Bro\\'n, Steven R, "Bibliography on Q Technique and Its 

Methodolo^^y", Perceptual and Motor Skills 587-613, 
1963, A fairly exhauGtive bibliography of the litera- 
ture on Q to its date (1968) including sections on So- 
cial Research using; Q. The basic bibliography, 

^- • * "On the Use of Variance Designs in Q 

Methodology", Psychological Recor d 20 i 179-189, 1970, 

5TButler, Jo.^n M, and Donald W, Fiske, "Theory and Tech- 
niques of Astteafiment" , Annual Review of Psycholo.^cv 
6» 327-35"^! (^^sp, pp. 331-333), 1955. 

6- Cattell, Riymond D, "The Three Basic Factor Analytic 

Research Designs--T}ieir Interrelations and Derivations", 
Ps.vcholo/?:ical Bulletin ^9*^99-520, 1962, 

7- Cronbach, Lee J, and Goldine Gleser. "Assessing Simi- 

larity Between Profiles", Psyoholog;ical Bulletin 50 1 
^56-^73, 1953. 

8- , Book Review of "The Study of 

Behavior". Psychometrika 19 i 327-330, 195^. 

9- Cronbach, Lae J, "Assessment of Individual Differences", 

Annual Re view of Psycholo,'z:y 7i I73-I96, (esp, p, I76), 
1956, 

10- Dahlstrom, W, Grant. "Personality", Annual Revi ew of 

Psycho.l.o.T y, 21 1 1-^8, 1970, 

11- Edv/ardSp Allen L, and Paul Horst, "Social Desirability 

as a Variable in Q Technique Studies", Educational 
and Psyc.'iolo^dcal Measurement 13 1^1 621-62^, 1953, 

12- Edwards, Allen L, "Social Desirability and Q Sorts", 

Journal of .Cons ulting?: Psycholor^y 19 1 6i 462, 1955, 

13- Eysenck, H,J, Book Review of "The Study of Behavior", 

Tho. Journal of gducational Fsycholoj;;.y ^-St 61 37^-376, 
19 5'^-. 
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l^-Frank, George H, "Note on the Reliability of Q-Sort 
Data", Psycholofiiioal R eport 2i 182, 1956. 

15- Gaito, John, "Forced and Free Q Sorts", Psycholo/^ical 

Report lOi 251-25^» 19^3. 

16- Goodling, Richard A, and George M, Guthrie, "Some 

Practical Considerations in Q-Sort Item Selection", 
Journal of C ounseling Psycholo.'ry 3» 1« 70-72, 1956, 

17- Guttman, Louis, "An Outline of Some New Methodology 

For Social Research", Public Opinion Quarterly 
18i 395-^0^4', (cf, n, 5)i 1955. 
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'APPENDIX C ^ 
.PROCEDURE AND INSTRUCTIONS FOR INTER-C0D3R RELIABILITY 



A Ph.D. student in communications was selected to 
be alternate coder. He was provided with the following in- 
structional materials I 

A) The methodological discucsion (pp. ^-22) with 
all references to articles-to-be-coded de- 
leted. 

B) The list of criteria (pp. 23 f f . ) 

C) Supplementary instructions for coding as follows i 
Criteria. II i 1. Use of a balanced block desi^ 
of some type v.'as accepccd as working to build 

In "mny staternents that do not matter" etc. on 
the basis of Stephenson's affirmation that such 
a design would usually take care of establishing 
the distensive zero of the sort. Basically-un- 
• structured sorts were taken as not working in 
this manner. 

Criteria IV t 1. In judging the articles the use 
of a non-dichotomous structured sort v/as consi- 
dered as providing "item-design support" for a 
forced bell-shaped distribution. 
Criteria Vi 2. When statistical tools were iden- 
tified only to the extent that intercorrolations 



and factor analysis were performed, for example, 
this was not considered sufficient since it is 
theoretically possible to use non-parametric 
factor analysis with Q and there are various ro- 
tational procedures etc» which might evoke dif- 
ferent kinds of factors. In any event replica- 
tions and data verification checks would not be 
possible with such skimpy information. 

The alternate coder was then provided with three non 

I 

communication Q articles for practice, 

! 

1) Marven 0, Nelson and Edward Morris, "An Applica- 
tion of the Q-Technique to the Study of Religious Concepts" 
Psychological Reports , 3 (1957 )i 293-''297. 

?.) Bernard Pyron* "Belief Q~Scrt, Allport-Vcmon 
Study of Values and Religion", PsycholOF^iccil Reports , 8 
(1961), 399-^00, 

3) William Van Dusen and William Rector, "A Q Sort 
Study of the Ideal Administrator ", Jo urnal of Olinical 
Psycholo^. 19, 2 (1963), 2^^, 

These were coded and then discussed with the author 
for clarity of criteria. The alternate coder was then as- 
signed the thirty articles for coding. His codings were 
compared to those of the authorc The results appear on 
p, 3 of the text. 



